Metaobject Protocols.html

   1 <?xml version="1.0" encoding="utf-8" ?>
   2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
   3     "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   4 <html xmlns="http://www.w3.org/1999/xhtml">
   5   <head>
   6     <title>Metaobject Protocols</title>
   7     <meta name="generator" content="muse.el" />
   8     <meta http-equiv="Content-Type"
   9           content="text/html; charset=utf-8" />
  10     <link href="http://feeds.unknownlamer.org/rss/site-updates"
  11           rel="alternate" type="application/rss+xml" title="Updates Feed" />
  12
  13 <link rel="stylesheet" href="default.css" />
  14   </head>
  15   <body>
  16     <h1>Metaobject Protocols</h1>
  17    <div class="contents">
  18 <dl>
  19 <dt>
  20 <a href="#sec1">Background</a>
  21 </dt>
  22 <dd>
  23 <dl>
  24 <dt>
  25 <a href="#sec2">Object Protocols</a>
  26 </dt>
  27 <dt>
  28 <a href="#sec3">CLOS Way of OO</a>
  29 </dt>
  30 <dd>
  31 <dl>
  32 <dt>
  33 <a href="#sec4">Classes for Scratch Data and Types</a>
  34 </dt>
  35 <dt>
  36 <a href="#sec5">Generics with Methods that Implement Protocols</a>
  37 </dt>
  38 </dl>
  39 </dd>
  40 </dl>
  41 </dd>
  42 <dt>
  43 <a href="#sec6">Limitations of Default Language Behavior</a>
  44 </dt>
  45 <dd>
  46 <dl>
  47 <dt>
  48 <a href="#sec7">Slot Storage</a>
  49 </dt>
  50 <dt>
  51 <a href="#sec8">Design Patterns</a>
  52 </dt>
  53 </dl>
  54 </dd>
  55 <dt>
  56 <a href="#sec9">Metasoftware</a>
  57 </dt>
  58 <dd>
  59 <dl>
  60 <dt>
  61 <a href="#sec10">Runtime Generated Classes</a>
  62 </dt>
  63 <dt>
  64 <a href="#sec11">Object Inspection</a>
  65 </dt>
  66 </dl>
  67 </dd>
  68 <dt>
  69 <a href="#sec12">Metaobject Protocols</a>
  70 </dt>
  71 <dd>
  72 <dl>
  73 <dt>
  74 <a href="#sec13">Limited/Generalized Internals of the Implementation</a>
  75 </dt>
  76 <dt>
  77 <a href="#sec14">Classes of MOPs</a>
  78 </dt>
  79 <dd>
  80 <dl>
  81 <dt>
  82 <a href="#sec15">Reflective</a>
  83 </dt>
  84 <dt>
  85 <a href="#sec16">Intercessory</a>
  86 </dt>
  87 </dl>
  88 </dd>
  89 <dt>
  90 <a href="#sec17">Violation of Encapsulation?</a>
  91 </dt>
  92 </dl>
  93 </dd>
  94 <dt>
  95 <a href="#sec18">MOP Design Principles</a>
  96 </dt>
  97 <dd>
  98 <dl>
  99 <dt>
 100 <a href="#sec19">Layered Protocol</a>
 101 </dt>
 102 <dd>
 103 <dl>
 104 <dt>
 105 <a href="#sec20">Top Level <strong>Must</strong> Call Lower Level Methods</a>
 106 </dt>
 107 <dt>
 108 <a href="#sec21">Lower Level Methods are Easier to Customize</a>
 109 </dt>
 110 </dl>
 111 </dd>
 112 <dt>
 113 <a href="#sec22">Functional Where Possible</a>
 114 </dt>
 115 <dd>
 116 <dl>
 117 <dt>
 118 <a href="#sec23">Memoization</a>
 119 </dt>
 120 <dt>
 121 <a href="#sec24">Constant Shared Return Values</a>
 122 </dt>
 123 </dl>
 124 </dd>
 125 <dt>
 126 <a href="#sec25">Procedural Only Where Necessary</a>
 127 </dt>
 128 <dt>
 129 <a href="#sec26">Real World</a>
 130 </dt>
 131 <dd>
 132 <dl>
 133 <dt>
 134 <a href="#sec27">UCW and Arnesi</a>
 135 </dt>
 136 <dt>
 137 <a href="#sec28">CLSQL</a>
 138 </dt>
 139 <dt>
 140 <a href="#sec29">Elephant</a>
 141 </dt>
 142 </dl>
 143 </dd>
 144 </dl>
 145 </dd>
 146 <dt>
 147 <a href="#sec30">Sources and Further Reading</a>
 148 </dt>
 149 <dd>
 150 <dl>
 151 <dt>
 152 <a href="#sec31">Sources</a>
 153 </dt>
 154 <dd>
 155 <dl>
 156 <dt>
 157 <a href="#sec32">The Art of the Metaobject Protocol</a>
 158 </dt>
 159 <dt>
 160 <a href="#sec33">CLOS MOP Specification</a>
 161 </dt>
 162 <dt>
 163 <a href="#sec34">Metaobject Protocols: Why We Want Them and What Else They Can Do</a>
 164 </dt>
 165 <dt>
 166 <a href="#sec35">Why Are Black Boxes so Hard to Reuse?</a>
 167 </dt>
 168 </dl>
 169 </dd>
 170 <dt>
 171 <a href="#sec36">Further Reading</a>
 172 </dt>
 173 <dd>
 174 <dl>
 175 <dt>
 176 <a href="#sec37">A Metaobject Protocol for C++</a>
 177 </dt>
 178 <dt>
 179 <a href="#sec38">Open Implementations and Metaobject Protocols</a>
 180 </dt>
 181 </dl>
 182 </dd>
 183 <dt>
 184 <a href="#sec39">Software</a>
 185 </dt>
 186 <dd>
 187 <dl>
 188 <dt>
 189 <a href="#sec40">Closer to MOP</a>
 190 </dt>
 191 </dl>
 192 </dd>
 193 </dl>
 194 </dd>
 195 </dl>
 196 </div>
 197
 198
 199 <!-- Page published by Emacs Muse begins here --><p>In Fall of 2006 I did a small project on Metaobject Protocols for my
 200 CS 331 class. Here lie my notes which may perhaps be useful to
 201 others. I hope to expand them into something more useful over time.</p>
 202
 203 <h2><a name="sec1" id="sec1"></a>
 204 Background</h2>
 205
 206 <h3><a name="sec2" id="sec2"></a>
 207 Object Protocols</h3>
 208
 209 <p class="first">An object protocol is a set of methods and specification of the
 210 interactions between the methods which provide some generic behavior
 211 (e.g. of a sequence) that are then implemented by classes which
 212 conform to the protocol (e.g. a vector or list). In most object
 213 systems a class contains both the methods which implement a protocol
 214 and the data used by the implementation. The intent is to emulate
 215 state machines which pass messages between each other.</p>
 216
 217
 218 <h3><a name="sec3" id="sec3"></a>
 219 CLOS Way of OO</h3>
 220
 221 <p class="first">The Common Lisp Object System (CLOS) is different. It separates
 222 the data and method concepts into classes and generics. A class
 223 contains data fields only, and a generic has methods specialized for
 224 certain types attached to it. This seems a bit weird at first, but is
 225 significantly more powerful as it encourages complete encapsulation
 226 through its use of classes primarily for method specialization rather
 227 than for state storage.</p>
 228
 229 <h4><a name="sec4" id="sec4"></a>
 230 Classes for Scratch Data and Types</h4>
 231
 232 <p class="first">In CLOS classes store data in slots (which are the same as data
 233 members). Encapsulation is not provided; any bit of code can use
 234 <code>slot-value</code> to access or set the value of a slot. This may seem odd at
 235 first, but encapsulation is of questionable importance as the slots
 236 are meant only to be used by the protocol defined around the class.</p>
 237
 238 <p>Classes are defined with <code>defclass</code></p>
 239
 240 <pre class="src">
 241 (<span class="emacs-face-keyword">defclass</span> <span class="emacs-face-type">name</span> (superclasses ...)
 242   ((slot-name <span class="emacs-face-builtin">:accessor</span> slot-accessor ...)
 243    ...)
 244   (class-options ...))
 245
 246 (<span class="emacs-face-keyword">defclass</span> <span class="emacs-face-type">example</span> ()
 247   ((foo <span class="emacs-face-builtin">:accessor</span> foo-of <span class="emacs-face-builtin">:initform</span> 5)))
 248
 249 (<span class="emacs-face-keyword">defclass</span> <span class="emacs-face-type">example-child</span> (example)
 250   ((bar <span class="emacs-face-builtin">:accessor</span> bar-of <span class="emacs-face-builtin">:initform</span> (list 1 2 3))))
 251 </pre>
 252
 253 <p>Slot definitions have several options; the above example shows only the
 254 <code>:accessor</code> and <code>:initform</code> options which are the most commonly
 255 used. <code>:accessor</code> generates an accessor for the slot (e.g. if you have
 256 an instance of <code>example</code> you can <code>(setf (foo-of some-example-instance)
 257 'some-value)</code> to set and <code>(foo-of some-example-instance)</code> to access the
 258 value). <code>:initform</code> provides a default initial value for the slot as a
 259 symbolic expression to be evaluated when an instance is created in the
 260 lexical environment of the class definition.</p>
 261
 262
 263 <h4><a name="sec5" id="sec5"></a>
 264 Generics with Methods that Implement Protocols</h4>
 265
 266 <p class="first">Generics are like normal functions in Lisp, but they only provide a
 267 lambda list (parameter list). Methods are added to the generic which
 268 specialize on the types of their parameters and provide an
 269 implementation. This allows writing rich layered protocols which can
 270 enable selective modification of individual facets with minimal code.</p>
 271
 272 <pre class="src">
 273 (<span class="emacs-face-keyword">defgeneric</span> <span class="emacs-face-function-name">generic</span> (parameters ...)
 274   (options) ...)
 275
 276 (<span class="emacs-face-keyword">defmethod</span> <span class="emacs-face-function-name">generic-name</span> ((parameter type) parameter ...)
 277   <span class="emacs-face-string">"documentation string"</span>
 278   body)
 279
 280 (<span class="emacs-face-keyword">defgeneric</span> <span class="emacs-face-function-name">foo</span> (bar baz quux)
 281   (<span class="emacs-face-builtin">:documentation</span> <span class="emacs-face-string">"Process the baz with the quux capacitor to make the
 282 foo widget fly into the sky at warp speed"</span>))
 283
 284 (<span class="emacs-face-keyword">defmethod</span> <span class="emacs-face-function-name">foo</span> ((bar example) baz (quux capacitor))
 285   (launch bar (process-with quux baz)))
 286 </pre>
 287
 288 <p>A method lambda list differs from a normal lambda list only in that it
 289 can specify the type of the parameter using the notation <code>(name type)</code>.
 290 Note also that methods can specialize on the types of every
 291 argument and not just the first one. This is quite powerful for
 292 reasons outside of the scope of this presentation.</p>
 293
 294
 295
 296
 297 <h2><a name="sec6" id="sec6"></a>
 298 Limitations of Default Language Behavior</h2>
 299
 300 <p class="first">The behavior of a language is a compromise between many competing
 301 issues that attempts to be as generally useful as possible so that
 302 <em>most</em> applications will have no issue with the default behavior. There
 303 are, however, certain applications that could be cleanly written with
 304 minor modifications to the behavior of the language, but would be
 305 impossible or quite difficult to write otherwise.</p>
 306
 307 <h3><a name="sec7" id="sec7"></a>
 308 Slot Storage</h3>
 309
 310 <p class="first">Most languages choose to preallocate storage for all of the slots of
 311 an instance. Now imagine a contact database that stores information
 312 about people in slots of a class. There may be dozens of slots, but
 313 often many of them will be left blank. If slot storage is preallocated
 314 much memory will be wasted and the database may not be able to fit
 315 into the memory of the hardware it must run on (perhaps for financial
 316 reasons, huge datasets, etc.).</p>
 317
 318 <p>To save memory the author of the contact database must implement his
 319 own system to store properties and allocate them lazily. This
 320 represents a fair bit of effort, and would implement a system that
 321 differed from the existing slot system of classes only regarding slot
 322 storage.</p>
 323
 324 <p>It would be useful if there were a way to customize slot allocation in
 325 instances. The customizations would be minor and require overriding
 326 only the initial allocation behavior and the behavior of the first
 327 assignment to the slot. It is a a trivial problem in a language that
 328 allows customization of these behaviors.</p>
 329
 330
 331 <h3><a name="sec8" id="sec8"></a>
 332 Design Patterns</h3>
 333
 334 <p class="first">Design Patterns are generalized versions of common patterns found in
 335 programs. Many of them are merely methods to get around deficiencies
 336 in the language, and can be quite messy to implement in some
 337 languages. Ideally a pattern would be subsumed by the language, but
 338 real world constraints require language standards to remain fairly
 339 static.</p>
 340
 341
 342
 343 <h2><a name="sec9" id="sec9"></a>
 344 Metasoftware</h2>
 345
 346 <p class="first">Some types of programs could be written easily if the language were
 347 customizable but are nearly impossible to write when it is not.</p>
 348
 349 <h3><a name="sec10" id="sec10"></a>
 350 Runtime Generated Classes</h3>
 351
 352 <p class="first">Say you wanted to write a video game where players could create their
 353 own objects, attach behaviors to the objects, and perhaps mix
 354 different objects together to create new ones. When you abstract the
 355 problem this looks just like an object system!  Wouldn't it be nice if
 356 your program could create new classes and methods on the fly portably?</p>
 357
 358
 359 <h3><a name="sec11" id="sec11"></a>
 360 Object Inspection</h3>
 361
 362 <p class="first">Imagine you were developing a complicated program with many different
 363 objects that interacted in fairly complex ways. A tool to inspect the
 364 structure of objects while debugging would be quite useful, but in a
 365 traditional language would be impossible to implement portably. This
 366 could force you to purchase a certain compiler implementation which
 367 provided an inspector, and even then would likely not be customizable.</p>
 368
 369 <p>This problem can be generalized to apply to most debugging tools; it
 370 would be useful to write such tools portably because users of the
 371 <em>language</em> and not the <em>compiler</em> need to debug software. Sharing
 372 infrastructure would result in better tools (more developers), and
 373 save the man-years of wasted effort that comes with having to rewrite
 374 unportable tools from scratch multiple times.</p>
 375
 376
 377
 378 <h2><a name="sec12" id="sec12"></a>
 379 Metaobject Protocols</h2>
 380
 381 <h3><a name="sec13" id="sec13"></a>
 382 Limited/Generalized Internals of the Implementation</h3>
 383
 384 <p class="first">A Metaobject Protocol (MOP) is a generalized and limited subset of the
 385 underlying language implementation. It is limited to allow multiple
 386 implementation strategies; this, along with careful design, is
 387 essential because programming language research is ever advancing and
 388 new techniques for creating more reliable and faster implementations
 389 are still being discovered.</p>
 390
 391 <p>This subset of the implementation is exported as a set of methods on
 392 metaobjects. Thus the language is implemented in itself. The system
 393 can then be customized using the extension and overriding features of
 394 the language itself.</p>
 395
 396
 397 <h3><a name="sec14" id="sec14"></a>
 398 Classes of MOPs</h3>
 399
 400 <h4><a name="sec15" id="sec15"></a>
 401 Reflective</h4>
 402
 403 <p class="first">A reflective MOP provides an interface to information <em>about</em> the
 404 running system. It exposes class relationships, the methods attached
 405 to a generic, etc. A reflective MOP often provides some functionality
 406 for creating new classes at runtime. Smalltalk was one of the first
 407 languages to expose a reflective MOP.</p>
 408
 409 <h5>Example: Object Inspector</h5>
 410
 411 <pre class="src">
 412 (<span class="emacs-face-keyword">defgeneric</span> <span class="emacs-face-function-name">example-inspect</span> (instance)
 413   (<span class="emacs-face-builtin">:documentation</span> <span class="emacs-face-string">"Simple object inspector using CLOS MOP"</span>))
 414
 415 (<span class="emacs-face-keyword">defmethod</span> <span class="emacs-face-function-name">example-inspect</span> ((instance t))
 416   (format t <span class="emacs-face-string">"Simple Object~% Value: ~S~%"</span> instance))
 417
 418 (<span class="emacs-face-keyword">defmethod</span> <span class="emacs-face-function-name">example-inspect</span> ((instance standard-object))
 419   (<span class="emacs-face-keyword">let</span> ((class (class-of instance)))
 420     (format t <span class="emacs-face-string">"Class: ~S, Superclasses: ~S~%"</span>
 421             (class-name class)
 422             (mapcar #'class-name
 423                     (class-precedence-list class)))
 424     (<span class="emacs-face-keyword">let</span> ((slot-names (mapcar #'slot-definition-name
 425                               (class-slots class))))
 426       (format t <span class="emacs-face-string">"Slots: ~%~{ ~S~%~}"</span> slot-names)
 427       (inspect-loop slot-names instance #'example-inspect))))
 428
 429 (<span class="emacs-face-keyword">defun</span> <span class="emacs-face-function-name">inspect-loop</span> (slots instance inspector)
 430   (format t <span class="emacs-face-string">"Enter slot to inspect or :pop to go up one level: "</span>)
 431   (finish-output)
 432   (<span class="emacs-face-keyword">let*</span> ((slot (read))
 433          (found-slot (member slot slots)))
 434     (<span class="emacs-face-keyword">cond</span> (found-slot
 435            (funcall inspector (slot-value instance slot))
 436            (funcall inspector instance))
 437           ((eq slot <span class="emacs-face-builtin">:pop</span>) t)
 438           (t
 439            (format t <span class="emacs-face-string">"~S is invalid. Valid slot names: ~S~%"</span>
 440                    slot
 441                    slots)
 442            (inspect-loop slots instance inspector)))))
 443 </pre>
 444
 445
 446 <h5>Example: Runtime Generated Classes and Methods</h5>
 447
 448
 449
 450 <h4><a name="sec16" id="sec16"></a>
 451 Intercessory</h4>
 452
 453 <p class="first">Intercessory MOPs allow the user to customize language behavior by
 454 implementing methods which override certain aspects of the language
 455 behavior. This class of MOPs are what make MOPs especially
 456 powerful. No longer must a problem be restructured to fit the
 457 implementation language; the underlying language can be reshaped to
 458 fit the task at hand, and obfuscation of the intended structure of the
 459 application can be avoided.</p>
 460
 461 <h5>Example: Lazily Allocated Slots</h5>
 462
 463
 464 <h5>Example: Observer Design Pattern</h5>
 465
 466 <p>A simple implementation of the observer pattern is under 100 lines,
 467 and the user level code requires only a single line of code to make
 468 any existing class observable.</p>
 469
 470 <p>In a language lacking a MOP, implementing the observer pattern
 471 requires modifying every accessor of a class to explicitly invoke any
 472 observers, and necessitates the addition of a mixin class to the class
 473 hierarchy. The fact that an object can be observed is a meta property
 474 of the class, and forcing it to be implemented at the application
 475 level dirties the inheritance hierarchy and adds unnecessary meta
 476 details to the program.</p>
 477
 478 <pre class="src">
 479 <span class="emacs-face-comment-delimiter">;;; </span><span class="emacs-face-comment">This metaclass adds a slot to instances which use it, and so the
 480 </span><span class="emacs-face-comment-delimiter">;;; </span><span class="emacs-face-comment">system is defined in its own package to avoid name conflicts
 481 </span>(<span class="emacs-face-keyword">defpackage</span> <span class="emacs-face-type">:observer</span>
 482   (<span class="emacs-face-builtin">:use</span> <span class="emacs-face-builtin">:cl</span> <span class="emacs-face-builtin">:c2mop</span>)
 483   (<span class="emacs-face-builtin">:export</span> observable register-observer unregister-observer))
 484
 485 (<span class="emacs-face-keyword">in-package</span> <span class="emacs-face-builtin">:observer</span>)
 486
 487 <span class="emacs-face-comment-delimiter">;;; </span><span class="emacs-face-comment">Metaclass
 488 </span>(<span class="emacs-face-keyword">defclass</span> <span class="emacs-face-type">observable</span> (standard-class)
 489   ()
 490   (<span class="emacs-face-builtin">:documentation</span> <span class="emacs-face-string">"Metaclass for observable objects"</span>))
 491
 492 (<span class="emacs-face-keyword">defmethod</span> <span class="emacs-face-function-name">compute-slots</span> ((class observable))
 493   <span class="emacs-face-string">"Add a slot for storing observers to observable instances"</span>
 494   (cons (make-instance 'standard-effective-slot-definition
 495                        <span class="emacs-face-builtin">:name</span> 'observers
 496                        <span class="emacs-face-builtin">:initform</span> '(make-hash-table)
 497                        <span class="emacs-face-builtin">:initfunction</span> #'(<span class="emacs-face-keyword">lambda</span> () (make-hash-table)))
 498         (call-next-method)))
 499
 500 (<span class="emacs-face-keyword">defmethod</span> <span class="emacs-face-function-name">validate-superclass</span> ((class observable)
 501                                 (super standard-class))
 502   t)
 503
 504 (<span class="emacs-face-keyword">defun</span> <span class="emacs-face-function-name">register-observer</span> (instance slot-name key closure)
 505   (register-observer-with-class (class-of instance)
 506                                 instance
 507                                 slot-name
 508                                 key
 509                                 closure))
 510
 511 (<span class="emacs-face-keyword">defun</span> <span class="emacs-face-function-name">unregister-observer</span> (instance slot-name key)
 512   (unregister-observer-with-class (class-of instance)
 513                                   instance
 514                                   slot-name
 515                                   key))
 516
 517 (<span class="emacs-face-keyword">defun</span> <span class="emacs-face-function-name">get-observers</span> (instance slot-name)
 518   (get-observers-with-class (class-of instance)
 519                             instance
 520                             slot-name))
 521
 522 (<span class="emacs-face-keyword">defun</span> <span class="emacs-face-function-name">add-observer-table</span> (instance slot-name)
 523   (setf (gethash slot-name (slot-value instance
 524                                        'observers))
 525         (make-hash-table)))
 526
 527 (<span class="emacs-face-keyword">defgeneric</span> <span class="emacs-face-function-name">register-observer-with-class</span> (class instance slot-name key closure))
 528 (<span class="emacs-face-keyword">defgeneric</span> <span class="emacs-face-function-name">unregister-observer-with-class</span> (class
 529                                             instance
 530                                             slot-name
 531                                             key))
 532
 533 (<span class="emacs-face-keyword">defmethod</span> <span class="emacs-face-function-name">register-observer-with-class</span> ((class observable)
 534                                          instance
 535                                          slot-name
 536                                          key
 537                                          closure)
 538   (setf (gethash key
 539                  (or (gethash slot-name
 540                               (slot-value instance 'observers))
 541                      <span class="emacs-face-comment-delimiter">;; </span><span class="emacs-face-comment">Lazily add observer hash tables
 542 </span>                     (add-observer-table instance slot-name)))
 543         closure))
 544
 545 (<span class="emacs-face-keyword">defmethod</span> <span class="emacs-face-function-name">unregister-observer-with-class</span> ((class observable)
 546                                            instance
 547                                            slot-name
 548                                            key)
 549   (remhash key (gethash slot-name
 550                         (slot-value instance 'observers))))
 551
 552 (<span class="emacs-face-keyword">defmethod</span> <span class="emacs-face-function-name">get-observers-with-class</span> ((class observable)
 553                                      instance
 554                                      slot-name)
 555   (gethash slot-name (slot-value instance 'observers)))
 556
 557 (<span class="emacs-face-keyword">defmethod</span> (<span class="emacs-face-function-name">setf slot-value-using-class)</span> <span class="emacs-face-builtin">:before</span> (new-value
 558                                                   (class observable)
 559                                                   instance
 560                                                   slot)
 561   (<span class="emacs-face-keyword">let</span> ((slot-name (slot-definition-name slot)))
 562     (<span class="emacs-face-keyword">if</span> (not (eq 'observers slot-name))
 563         (<span class="emacs-face-keyword">let</span> ((observers
 564                (get-observers instance (slot-definition-name slot))))
 565           (<span class="emacs-face-keyword">if</span> observers
 566               (maphash #'(<span class="emacs-face-keyword">lambda</span> (key observer)
 567                            (funcall observer
 568                                     (<span class="emacs-face-keyword">if</span> (slot-boundp instance slot-name)
 569                                         (slot-value instance slot-name)
 570                                       nil)
 571                                     new-value))
 572                        observers))))))
 573 </pre>
 574
 575
 576
 577
 578
 579 <h3><a name="sec17" id="sec17"></a>
 580 Violation of Encapsulation?</h3>
 581
 582 <p class="first">A MOP may seem like a violation of encapsulation by revealing some
 583 implementation details, but in reality a well designed protocol does
 584 not reveal anything which was not already exposed. Implementation
 585 decisions affect users, and some of these details do leak through to
 586 higher levels (e.g. the memory layout of slots). Implicit in the
 587 protocol specification are these implementation details, and the MOP
 588 merely makes this limited subset available for customization.</p>
 589
 590 <p>A MOP makes it possible to customize certain implementation decisions
 591 that do not <strong>radically</strong> alter the behavior of the base language. The
 592 conceptual vocabulary of the system retains its meaning, and so code
 593 written in one dialect can interact with code written in another
 594 without knowing that they speak different ones.</p>
 595
 596
 597
 598 <h2><a name="sec18" id="sec18"></a>
 599 MOP Design Principles</h2>
 600
 601 <h3><a name="sec19" id="sec19"></a>
 602 Layered Protocol</h3>
 603
 604 <p class="first">A layered protocol design is good for both meta and normal object
 605 protocols, and enables a combinatorial explosion of customizations to
 606 the protocol.</p>
 607
 608 <h4><a name="sec20" id="sec20"></a>
 609 Top Level <strong>Must</strong> Call Lower Level Methods</h4>
 610
 611 <p class="first">The top level methods of a layered protocol are required to call
 612 certain lower level methods to perform some tasks. This both makes it
 613 easier to customize the top level methods (which perform very broad
 614 tasks) by providing some pieces of implementation for the programmer,
 615 and enables more customization by opening up the replacement of lower
 616 level functions as a way to alter a small detail of the high level
 617 behavior.</p>
 618
 619
 620 <h4><a name="sec21" id="sec21"></a>
 621 Lower Level Methods are Easier to Customize</h4>
 622
 623 <p class="first">The lower level methods of a MOP are limited in scope and can be
 624 implemented easily. Often the desired changes to language behavior are
 625 minor, and having methods that perform simple tasks which are often
 626 customized reduces the effort required to extend the system.</p>
 627
 628
 629
 630 <h3><a name="sec22" id="sec22"></a>
 631 Functional Where Possible</h3>
 632
 633 <p class="first">Functional protocols are preferred for MOPs (and object protocols in
 634 general). Functional protocols open up several optimizations for the
 635 implementation without burdening the user of the protocol.</p>
 636
 637 <h4><a name="sec23" id="sec23"></a>
 638 Memoization</h4>
 639
 640 <p class="first">Memoization is the process of saving the results of a function call
 641 for future use. This avoids expensive recomputation of values which
 642 have not changed (recall that a true function will always return the
 643 same result when given the same arguments).</p>
 644
 645 <p>A functional MOP can be optimized easily by exploiting this property
 646 to memoize the return values of calls to expensive operations. A MOP
 647 must be be very fast to avoid making programs unusably slow, and
 648 memoization is able to give an appreciable speedup in many cases
 649 without a significant burden on memory usage.</p>
 650
 651
 652 <h4><a name="sec24" id="sec24"></a>
 653 Constant Shared Return Values</h4>
 654
 655 <p class="first">Disallowing modification of values returned by protocol methods allows
 656 the implementation to return large data structures by reference to
 657 avoid expensive copying without having to do expensive data integrity
 658 checks or copying.</p>
 659
 660
 661
 662 <h3><a name="sec25" id="sec25"></a>
 663 Procedural Only Where Necessary</h3>
 664
 665 <p class="first">Some operations like method invocation are inherently stateful and so
 666 must use a procedural protocol. There is no benefit to be gained from
 667 using a functional protocol, and indeed an attempt would result in
 668 obtuse code that severely restricted the implementation. Do note that
 669 only a very small part of method invocation is stateful (the actual
 670 call), and most of it can be implemented functionally (e.g. computing
 671 the discriminating function).</p>
 672
 673
 674 <h3><a name="sec26" id="sec26"></a>
 675 Real World</h3>
 676
 677 <h4><a name="sec27" id="sec27"></a>
 678 <a href="http://common-lisp.net/project/ucw/">UCW</a> and <a href="http://common-lisp.net/project/bese/arnesi.html">Arnesi</a></h4>
 679
 680 <p class="first">Arnesi uses the CLOS MOP to implement methods which are transparently
 681 rewritten into continuation passing style. This allows their execution
 682 to be suspended at certain points and resumed later. UCW builds on top
 683 of this to support a web framework where the statelessness of http is
 684 hidden from the user; displaying a page suspends the execution of the
 685 current continuation, and resumes it upon submission. The user level
 686 code is completely unaware of this.</p>
 687
 688
 689 <h4><a name="sec28" id="sec28"></a>
 690 <a href="http://clsql.b9.com">CLSQL</a></h4>
 691
 692 <p class="first">CLSQL uses the reflective part of the CLOS MOP to map Common Lisp data
 693 types into SQL types, and the intercessory protocol for slot
 694 allocation to map slots onto database columns or sql expressions (for
 695 implementing relational slots).</p>
 696
 697
 698 <h4><a name="sec29" id="sec29"></a>
 699 <a href="http://common-lisp.net/project/elephant/">Elephant</a></h4>
 700
 701 <p class="first">Elephant uses the CLOS MOP to transparently store any class to disk
 702 and handle paging between the disk store and memory efficiently
 703 without user intervention.</p>
 704
 705
 706
 707
 708 <h2><a name="sec30" id="sec30"></a>
 709 Sources and Further Reading</h2>
 710
 711 <h3><a name="sec31" id="sec31"></a>
 712 Sources</h3>
 713
 714 <h4><a name="sec32" id="sec32"></a>
 715 The Art of the Metaobject Protocol</h4>
 716
 717 <h5>Kiczales, Gregor et al. MIT Press 1991</h5>
 718
 719 <p>Highly recommended reading even if you plan to never implement a MOP
 720 or use the CLOS one. The design principles it recommends are quite
 721 useful.</p>
 722
 723
 724
 725 <h4><a name="sec33" id="sec33"></a>
 726 <a href="http://www.lisp.org/mop/contents.html">CLOS MOP Specification</a></h4>
 727
 728 <p class="first">Specification of the MOP for CLOS defined in <em>The Art of the Metaobject Protocol</em>.</p>
 729
 730
 731 <h4><a name="sec34" id="sec34"></a>
 732 <a href="http://citeseer.ist.psu.edu/399658.html">Metaobject Protocols: Why We Want Them and What Else They Can Do</a></h4>
 733
 734 <p class="first">A short overview of MOP design principles followed by three example
 735 metaobject protocols for Scheme.</p>
 736
 737
 738 <h4><a name="sec35" id="sec35"></a>
 739 <a href="http://www2.parc.com/csl/groups/sda/projects/oi/towards-talk/transcript.html">Why Are Black Boxes so Hard to Reuse?</a></h4>
 740
 741 <p class="first">Transcription of a talk on the benefits of open implementations of
 742 software. It first discusses several problems that black box software
 743 implementations pose, and then presents existing solutions. It shows
 744 how the existing solutions are insufficient, and then provides
 745 metaobject protocols as a solution to most of the problems.</p>
 746
 747
 748
 749 <h3><a name="sec36" id="sec36"></a>
 750 Further Reading</h3>
 751
 752 <h4><a name="sec37" id="sec37"></a>
 753 <a href="http://citeseer.ist.psu.edu/chiba95metaobject.html">A Metaobject Protocol for C++</a></h4>
 754
 755 <p class="first">Example of a purely compile time MOP. It implements the functionality
 756 of a code walker and something similar to the Lisp macro system.</p>
 757
 758
 759 <h4><a name="sec38" id="sec38"></a>
 760 <a href="http://www.parc.com/csl/groups/sda/publications/papers/Kiczales-TUT95/for-web.pdf">Open Implementations and Metaobject Protocols</a></h4>
 761
 762 <p class="first">It is a bit long, but it seems to follow a similar structure to AMOP
 763 in introducing MOPs and their usefulness. The pages are slides with
 764 notes, and so the 331 pages might not actually take that long to read.</p>
 765
 766
 767
 768 <h3><a name="sec39" id="sec39"></a>
 769 Software</h3>
 770
 771 <h4><a name="sec40" id="sec40"></a>
 772 <a href="http://common-lisp.net/project/closer/closer-mop.html">Closer to MOP</a></h4>
 773
 774 <p class="first">Compatibility layer that attempts to present the <em>Art of the Metaobject
 775 Protocol</em> MOP specification properly in as many Common Lisp
 776 implementation as possible.</p>
 777
 778
 779
 780
 781   <!-- Page published by Emacs Muse ends here -->
 782
 783   <p class="cke-buttons">
 784     <!-- validating badges, any browser, etc -->
 785     <a href="http://validator.w3.org/check/referer"><img
 786         src="http://www.w3.org/Icons/valid-xhtml10"
 787         alt="Valid XHTML 1.0!" /></a>
 788
 789     <a href="http://www.anybrowser.org/campaign/"><img
 790         src="img/buttons/w3c_ab.png" alt="[ Viewable With Any Browser
 791         ]" /></a>
 792
 793     <a href="http://www.debian.org/"><img
 794         src="img/buttons/debian.png" alt="[ Powered by Debian ]" /></a>
 795
 796     <a href="http://hcoop.net/">
 797       <img src="img/buttons/hcoop.png"
 798         alt="[ Hosted by HCoop]" />
 799     </a>
 800
 801     <a href="http://www.fsf.org/register_form?referrer=114">
 802       <img src="img/buttons/fsf_member.png"
 803         alt="[ FSF Associate Member ]" />
 804     </a>
 805   </p>
 806
 807 <p class="cke-footer">                         Ruled by the ebb of my oceans
 808                         Slaves to the dusk and the dawn
 809                          Your petri dish civilisations
 810                              Are buried and born
 811 </p>
 812 <p class="cke-timestamp">Last Modified:
 813     January 30, 2009</p>
 814   </body>
 815 </html>